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Note: In this brief, the term "early childhood education" (ECE) is used to refer to the teaching and learning of 
toddlers and preschool-age children informal care settings, whether they be group, family, center, or school- 
based. 


Introduction 

Early childhood education has emerged as a priority for national, state, and local lawmakers and enjoys 
broad bipartisan support. In 2015 alone, 22 states with Republican governors and 10 states with 
Democratic governors expanded access to early education programs, increasing state investment 
nationally by 12 percent in a single year.i 


Despite being one of the first states in the country to 
invest in public pre-k and establish quality 
standards for early education, Pennsylvania has 
fallen behind in providing equity and access to high 
quality education in the early years. In 2015, 
Education Week published its annual Early 
Education Index, which ranked Pennsylvania 415^ 
nationally, well behind our neighbors in New York, 
New Jersey, and Delaware.^ 

Citing these challenges. Governor Tom Wolf ran on 
the campaign promise of universal pre-k access, and 
proposed an unprecedented budget increase for 
early childhood programs in 2016. In Philadelphia, 
Mayor Jim Kenney has made pre-k a cornerstone of 
his education agenda and Comcast CEO David Cohen 
has called on the business community to invest in 
this "critical equalizer” that can "turn around our 
schools and support our city and our economy for 
long-term growth.”^ This call to action comes with 


Key Findings 

• Quality Ratings and Improvement Systems 
(QRIS) not only evaluate and rate early 
childhood education programs, they also 
provide resources and targeted supports 
aimed at improving program quality. 

• Promising practices in other states can help 
Pennsylvania better align their QRIS 
standards and supports with what matters 
most for child outcomes. 

• The state should consider ways to make staff 
requirements more flexible, provide coaching 
and technical assistance to more providers, 
and invest in programs like T.E.A.C.H. to 
increase the pipeline of qualified early 
educators. 

• If policymakers want to see sustainable 
expansion of high quality programs, they will 
also have to address the current gap between 
reimbursement rates and the true cost of 
providing quality care and education. 


^ Parker, E., Atchison, B. & Workman, E. (January 2016). State pre-k funding for 2015-16 fiscal year: National trends in state preschool funding. 
Education Commission of the States 50-State Review. Retrieved from http://www.ecs.org/ec-content/uploads/01252016_Prek-K_Funding_report- 
4.pdf 

2 http://www.edweek.org/media/education-week-earlv-education-index-2015.pdf 

3 Cohen, D.L. (2016, February 29). Philly businesses should back Kenney on pre-K funding. Philadelphia Inquirer. Retrieved from 
http://mobile.philly.com/beta?wss=/philly/blogs/thinktank&id=370507931#Yidx0YlJbDKYuhyk.99 
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good reason; as outlined in a previous PACER brief, research clearly documents the benefits of high-quality 
ECE4 

Yet research is equally clear that not all ECE programs are created equal. Thus, as policy-makers push for 
expanded access to early childhood education, there is a real need to ensure capacity to serve children in 
high-qualily settings. Systems that support quality improvement must define quality in a meaningful and 
rigorous way while also being responsive to the needs of a diverse set of early childhood education 
providers. To this end, the Pennsylvania Office of Child Development and Early Learning [OCDEL) has 
embarked on an effort to refine Keystone STARS, the state's early childhood education Quality Ratings and 
Improvement System [QRIS), to boost program participation and impacts. 

In this brief Research for Action explores existing research on the "quality characteristics" of ECE 
programs that improve child outcomes and the ways in which a QRIS can be designed to evaluate and 
promote these quality characteristics. We share innovative practices from other states' QRISs that may 
prove valuable as OCDEL strives to refine Keystone STARS. Finally, we highlight important policy 
considerations for local and state leaders as they seek to expand access to quality early learning programs. 


Quality Ratings and Improvement Systems 

Nationwide, an increasing number of states have outlined standards and designed QRIS logic models to 
support and evaluate the quality of ECE programs. Many states either require or incentivize ECE providers 
to participate in their QRIS. Today, 40 states and the District of Columbia have a statewide QRIS, nearly 
double the number from five years ago.^ At least five more are in the process of planning or piloting such a 
system.*’ 

Pathways to Quality Improvement and School Readiness 

The goals and corresponding design of QRISs vary by state. But most logic models consider preparing 
children for "school readiness” as the desired outcome and pursue two major pathways toward that goal: 

1. Targeted support for programs: QRISs provide resources, technical assistance, and 
professional development to help improve program quality in ECE. 

2. Program evaluation: QRISs regularly evaluate, rate, and monitor ECE providers for 
compliance with state standards. 


Figure 1 depicts these QRIS pathways to quality. 


Park, E. & Sludden, J. (April 2014). PACER Issue Brief: Early Childhood Education in Pennsylvania. Research for Action. Retrieved from 
http://8rri53pm0cs22jk3vvqnalub-wpengine.netdna-ssl.eom/wp-content/uploads/2015/10/RFA_PACER_Early_Childhood_Ed_April_2014.pdf 
5 Barnett (2011). Preschool Education as an Educational Reform: Issues of Effectiveness and Access. New Brunswick, NJ: National Institute for Early 
Education Research, Rutgers University; National Association for the Education of Young Children (2010). The NAECY QRIS Toolkit. Washington, DC: 
NAECY. 

® Flolod, A., Faria, A.M., Weinberg, E. & Floward, E. (2015). Moving Up the Ladder: How Do States Deliver Quality Improvement Supports Within Their 
Quaiity Rating and Improvement Systems? Washington, DC: American Institutes for Research. Retrieved from 

http://www.air.org/sites/default/files/downloads/report/Oualitv-Rating-and-lmDrovement-Svstems-ORIS-Earlv-Childhood-Sept-2015rev.pdf 
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Figure 1: Pathways to Quality Improvement & School Readiness 



As depicted in Figure 1, the two core QRIS functions, support and evaluation, drive improved program 
design and delivery by supporting and incentivizing providers to develop practices that have demonstrated 
impacts on school readiness, and other child outcomes. In addition, QRIS ratings serve as markers of 
quality that can help parents become more informed "consumers".^ A strong QRIS becomes a continuous 
loop that evaluates, supports, and drives demand for quality. 

Quality Characteristics and Measures 

It’s important to note that for this to model to work effectively, the QRIS must correctly identify the quality 
characteristics that increase school readiness and help provide the resources and conditions that produce 
and sustain continuous program improvement in those areas. 

Researchers have identified several important characteristics that contribute to program quality and thus 
improve child outcomes. 

These "quality characteristics” include: 

• Quality adult-child interactions. Recent advances in neuroscience have led to breakthroughs in our 
understanding of child development, and the ways in which responsive adult-child interactions 
actually build neural connections in the young brain and strengthen life-long communication skills and 


'< Zellman, G.L., Perlman, M., Le, V-N., Setodji, C.M. (2008). Assessing the Validity of the Qualistar Early Learning Quality Rating and Improvement 
System as a Tool for Improving Child-Care Quality. Santa Monica, CA: RAND Corporation. Retrieved from 
http://www.rand.ore/content/dam/rand/pubs/monograDhs/2008/RAND MG650.pdf 
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mental health.^ Research has also demonstrated that secure relationships between early childhood 
teachers and young children promote improved behavior and peer relationships later in life.^ 

• Accountability and support focused on instruction. Systems that clearly articulate program 
goals and provide teachers with the support to meet them facilitate effective teaching.io In 
particular, technical assistance and teacher mentoring to improve instruction have demonstrated 
positive effects on programs and participants. In one study, researchers found that youth taught by 
teachers who participated in REDI — a Head Start intervention that provided teachers with 
"enhanced professional development” — had higher learning engagement and social problem- 
solving skills with reduced aggressive-disruptive behavior compared to their non-REDI peers. n 
Similarly, a Child Trends study of Success By 6 's intensive coaching and support programming in 
Philadelphia found that participating centers were more likely to increase their quality rating level 
than similar, non-participating centers.12 

• Strong, coherent curriculum. While the debate about the ideal balance of academics and play- 
based exploration in early childhood classrooms continues, experts agree that, with regard to 
structured learning time, the most effective curricula focus on specific learning goals — cognitive, 
social, emotional, and physical — and then identify planned activities which are linked to these 
goals, as well as to daily schedules and routines, and the materials to be used.i^ 

• Teacher education. Teacher education is a necessary but not sufficient indicator of teacher quality. 
The limited rigorous research on the relationship between staff qualifications, quality indicators, 
and child outcomes in ECE yields mixed results. Several studies have found that teachers with 
higher education levels scored higher on observations of adult-child interactions, and in particular, 
that teachers with associates degrees and CDA certificates were more effective than teachers with 
only some college.isie Another found that preschoolers whose teachers had higher education levels 
exhibited slightly stronger basic reading and math skills.i^ However, others have failed to find any 
positive correlations.!^ Renowned National Institute for Early Education Research researcher Steve 
Barnett pointed out that wide variation exists in the quality of early childhood degree programs and 
thus makes teachers' highest level of education an inherently inadequate measure of teacher 


http://developingchild.harvard.edu/resources/inbrief-science-of-ecci/; Hamre, B., Hatfield, B., Pianta, R. & Jamil, F. (2013). Evidence for general and 
domain-specific elements of teacher-child interactions: Associations with preschool children’s development. Child Development, 85(3), 1257-1274. 

^ Elicker, J. & Fortner-Wood, C. (1995). Adult-child relationships in early childhood programs. Young Children 51(1), 69-78. 

“Welsh et al. (2010). The development of cognitive skills and gains in academic school readiness for children from low-income families. Journal of 
Educational Psychology, 103(11), 43-53 as cited in Barnett (2011). Preschool Education as an Educational Reform: Issues of Effectiveness and 
Access. 

“ Bierman, K.L., Nix, R.L, Heinrichs, B.S., Domitrovich, C.E., Gest, S.D., Welsh, J.A. & Gill, S. (2014). Effects of Head Start REDI on children's 
outcomes one year later in different kindergarten contexts. Child Development, 85(1), 140-159. 

“ Warner-Richter, M., Lowe, C., Tout, K., Epstein, D. & Li, W. (2016). Improving Quality for Child Care Centers in Greater Philadelphia: An Evaluation of 
Success By 6. Washington, DC: Child Trends. 

“ National Institute for Early Education Research (2016). It’s in the Stars: More States are Using Quality Rating Systems for Pre-K. New Brunswick, 

NJ: National Institute for Early Education Research, Rutgers University. Retrieved from http: / /nieer.org/publications/it%E2%80%99s-stars-more- 
states-are-usinu-qualitv-rating-svstenis-pre-k : National Association for the Education of Young Children & National Association of Early Childhood 
Specialists In State Departments of Education (2003). Early Childhood Curriculum, Assessment, and Program Evaluation: Building an effective, 
accountable ^stem in programs for children birth through age 8. Washington, DC: National Association for the Education of Young Children. 
Retrieved from httD://www.naevc.org/flles/naevc/flle/Dosltlons/CAPEexDand.Ddf 

“ Barnett (2011). Preschool Education as an Educational Reform: Issues of Effectiveness and Access. New Brunswick, NJ: National Institute for Early 
Education Research, Rutgers University. 

“ Pianta, R., Howes, C., Burchinal, M., Bryant, D., Clifford, R., Early, D. & Barbarin, 0. (2005). Features of pre-kindergarten programs, classrooms, and 
teachers: Do they predict observed classroom quality and child-teacher interactions? Applied Developmental Science, 9(3), 144-159. 

“ Howes, C. (1997). Children's experiences in center-based child care as a function of teacher background and adult:child ratio. Merrill-Palmer 
quarterly (Wayne State University. Press) 43:404-425 

“ Early, D.M., Bryant, D.M., Pianta, R.C., Clifford, R.M., Burchinal, M.R., Ritchie, WS., Howes, C. & Barbarin, 0. (2006). Are teachers’ education, major, 
and credentials related to classroom quality and children’s academic gains in pre-kindergarten? Early Childhood Research Quarterly, 21(2), 174-195. 
“ Ibid. 
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quality.15 In fact, research shows that other factors, such as compensation and ongoing professional 
development, have a greater overall impact on teacher professionalism .20 Nevertheless, credentials 
and degrees serve as an easily measurable indicator of experience and provide a minimum 
standard for pre-service training. 

• Smaller class sizes and low child-to-teacher ratios. Research confirms that children in smaller 
early childhood classes with lower child-to-teacher ratios exhibit better cognitive and social skills .21 
This is not surprising given that these measures serve as proxies for quality and frequency of adult- 
child interactions. 

• High Dosage. Research has demonstrated that full-day programs have greater impact on children's 
socio-emotional, language, math, and physical development than half-day programs .22 Extended day 
and extended year programs also produce greater learning gains over part-time or short-term 
programs.23 

• Family Engagement. A growing body of research highlights the important role of family 
engagement in early education. A 2011 study found that families whose children attended Head 
Start increased their activities to support child development at home during and after participation 
in the program .24 What's more, research has demonstrated positive effects of parent engagement in 
early childhood on a number of social and academic competencies in young children.^s 

• Adequate Funding. Research has not directly demonstrated a causal relationship between levels of 
funding for early education and child outcomes. However, oft-cited program evaluations of the 
Abecedarian Project and Perry Preschool showed the most robust effects on student outcomes to 
date, and these programs cost over $18,000 and $12,000 per child in 2016 dollars respectively.^^ 
Both were able to offer extended services, such as health screenings, recruit staff with certification 
or higher degrees, maintain small class sizes, and build robust family engagement. In comparison, 
Tennessee's public pre-k program, whose 2015 evaluation caused controversy when it revealed no 
lasting impact, cost approximately $4,611 per child in 2014.27 

These quality components are widely agreed-upon, but as described below, how a program or state system 
defines and measures them to quantify the success of a QRIS can be complex. 


Barnett (2011). Preschool Education as an Educational Reform: Issues of Effectiveness and Access. New Brunswick, NJ: National Institute for Early 
Education Research, Rutgers University. 

20 Burchinal, M., Hyson, M., & Zaslow, M. (2011). Competencies and credentials for early childhood educators: What do we know and what do we 
need to know. In E. Zigler, W. S. Gilliam & W. S. Barnett (Eds.) The pre-k debates: Current controversies & issues (pp. 73-77). Baltimore, MD: Brookes 
Publishing.; Pianta, R.C., Barnett, W.S., Burchinal, M. & Thornburg, K.R. (2011). The effects of preschool education: What we know, how public policy 
is or is not aligned with the evidence base, and what we need to know. Psychological Science in the Public Interest 10(2), 49-88. 

21 Ruopp, R. (1979). Children at the center: Summary findings and their implications. Final Report of the National Day Care Study, Volume I. 
Cambridge, MA: Abt Associates, Inc. Retrieved from http://eric.ed.gov/?id=ED1687 33 

22 Reynolds, A.J., Richardson, B.A., Hayakawa, M., Lease, E.M., Warner-Richter, M., England, M.M., Ou, S.R., & Sullivan, M. (2014). Association of a 
full-day vs part-day preschool intervention with school readiness, attendance, and parent involvement. The Journal of the American Medical 
Association, 312(20): 2126-2134. 

23 Robin, K.B., Frede, E.C. & Barnett, W.S. (2006). Is More Better? The Effects of Full-Day vs. Half-Day Preschool on Early School Achievement. New 
Brunswick, NJ: National Institute for Early Education Research, Rutgers University. Retrieved from 
http://nieer.org/resources/research/lsMoreBetter.pdf 

2^* Gelber, A.M. & Isen, A. (2011). Children’s schooling and parents’ investment in children: Evidence from the Head Start Impact study (Working Paper 
No. 17704). Retrieved from http://www.nber.org/papers/wl7704 

25 McWayne, C., Fantuzzo, J., Cohen, H. L., & Sekino, Y. (2004). A multivariate examination of parent involvement and the social and academic 
competencies of urban kindergarten children. Psychology in the Schools, 41(3), 363-377. Retrieved from 

https://www.researchgate.net/publication/229662621 A multivariate examination of parent involvement and the social and academic compe 

tencies of Urban Kindergarten 

25 Masse, L. N., & Barnett, W. S. (2002). A benefit-cost analysis of the Abecedarian early childhood intervention. Cost-Effectiveness and Educational 
Policy, Larchmont, NY: Eye on Education, Inc, 157-173. Retrieved from http://nieer.org/resources/research/AbecedarianStudv.pdf 
21 Barnett, W.S., Carolan, M.E., Squires, J.H., Clarke Brown, K., & Horowitz, M. (2015). The state of preschool 2014: State preschool yearbook. New 
Brunswick, NJ: National Institute for Early Education Research. Retrieved from http://www.nieer.org/sites/nieer/files/Tennessee 2014 O.pdf 
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Evaluations of Quality Ratings and Improvement System Impacts: A Mixed Bag 

Despite widespread acceptance of QRIS logic models and the rapid expansion of QRISs in recent years, 
there is limited research to demonstrate if, and how, QRISs themselves impact child outcomes. Studies have 
instead tended to focus on questions of design and implementation, and on validating particular measures 
of quality. However, three rigorous third-party evaluation studies of state QRISs in California, Colorado, and 
Pennsylvania have explored, or are in the process of exploring, the impact of those systems on child 
outcomes. These evaluations were each limited by data availability and the lack of strong metrics with 
which to assess certain aspects of quality. We summarize each below. 

California. California's evaluation of its QRIS is on-going, and data connecting the QRIS program ratings to 
child outcomes are not yet available. However, an initial validation and implementation study conducted by 
the American Institutes for Research (AIR) and RAND provided evidence that QRIS ratings in that state are 
significantly, positively correlated with three individual measures of quality: (1) adult-child interaction 
scores as measured by the Preschool Program Quality Assessment (PQA), (2) instructional support scores 
as measured by Classroom Assessment Scoring System (CLASS), and (3) scores on the Environmental 
Rating Scale (ERS).28 It is notable that these measures seek to quantify the three components of an ECE 
program - relationships, instruction, and class-size - that research suggests exert the greatest impact on 
child outcomes. 

Colorado. In 2008, researchers from RAND published findings from an evaluation of Colorado's 
"Qualistar" Early Learning QRIS, which examined the relationship between ratings and child outcomes. 
They found little evidence that the QRIS rating or individual quality components were predictive of child 
outcomes. 29 Further, while provider quality had improved over time, researchers could not unequivocally 
attribute that improvement to participation in the QRIS. 

Pennsylvania. In Pennsylvania, results of a 2015 third-party evaluation were somewhat more promising. 
Researchers from the Penn Child Research Center and the Consortium for Policy Research in Education 
(CPRE) found that children in high quality (Keystone STAR 3 and 4) centers had better child outcomes, as 
measured by Work Sampling System (WSS) scores, than children in STAR 1 and 2 centers, Although effect 
sizes were small, these findings provide some support for the common QRIS logic model, which holds that 
participation in QRIS drives the quality of classroom environment and adult-child interactions, which in 
turn impacts child development and school readiness. However, CPRE's inquiry found no substantive 
difference in child outcomes between STAR 1 and STAR 2 centers, or between STAR 3 and STAR 4 
centers.2i This suggests that the most important jump in quality may occur when providers move from 
STAR 2 to STAR 3. Further research is needed to determine why that is and how to best align standards and 
requirements to make progress through all STARS levels more meaningful. 

Summary. Taken together, these evaluations provide mixed results. California's evaluation demonstrates 
that robust QRIS systems do correlate with some of the most common, validity-tested quality metrics, such 
as Environmental Rating Scales (such as ECERS) and observation tools (such as PQAs or CLASS), which 
themselves are moderately predictive of child outcomes. Pennsylvania's evaluation provides evidence of a 
small but more direct relationship between QRIS ratings themselves and child outcomes. However, 


Hawkinson, L.E., Quick, H.E., Muenchow, S., Anthony, J., Weinberg, E., Hoiod, A Karoiy, LA. (2015). Independent Evaluation of California’s Race 

to the Top- Early Learning Challenge Quality Rating and Improvement System: Half-Term Report. Washington, DC: American institutes for Research. 
Retrieved from http://www.cde.ca.gov/sD/cd/rt/documents/airhalftermreport.pdf 
Zellman, G.L., Perlman, M., Le, V-N., Setodji, C.M. (2008). Assessing the Validity of the Qualistar Early Learning Quality Rating and Improvement 
System as a Tool for Improving Child-Care Quality. Santa Monica, CA: RAND Corporation. Retrieved from 
http://www.rand.org/content/dam/rand/pubs/monographs/2008/RAND MG650.pdf 
Sirinides, P., Fantuzzo, J., LeBoeuf, W., Barghaus, K., & Fink, R. (2015). An inquiry into Pennsylvania's Keystone STARS. Philadelphia, PA: 
Consortium for Policy Research in Education. Retrieved from http://www.cpre.0rg/sites/default/files/stars_inquiry_report_l.6.l6_final.pdf 
Ibid. 
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Colorado's evaluation found no such relationship. What's more, all three studies demonstrate that QRIS 
rubrics often include standards with little proven correlation with child outcomes. In part, this may reflect 
poor design of the QRIS logic model. It likely also reflects the reality that certain structural components of 
ECE programs, such as business practices and director credentials, may be important for program 
sustainability, but do not directly influence child outcomes. For this reason, it is essential that states 
continue to improve alignment between their QRISs and the research on what matters most for child 
outcomes. 


Pennsylvania’s Keystone STARS 

In 2003, Pennsylvania's Office of Child Development and Early Learning [OCDEL) introduced Keystone 
STARS [Standards, Training/Professional Development, Assistance, Resources, and Support), a voluntary 
QRIS, to assess and support the quality of early childhood programs across the state. The creation of 
Keystone STARS, one of the first tiered quality rating systems for early education programs in the nation, 
was in part a response to a 2002 report which highlighted the decreasing quality of childcare in 
Pennsylvania over nearly two decades.^z 

Programs opting to participate in Keystone STARS are evaluated based on a series of performance 
standards, which are grouped into four key areas: 

1) Staff Qualifications and Professional Development; 

2) Learning Program; 

3) Family and Community Partnerships; and 

4) Leadership and Management. 

Based on a program's ability to meet performance standards in each of these areas, they are assigned a 
rating from STAR 1 to STAR 4, with STAR 4 being the highest and most rigorous quality designation. STAR 
1 and 2 programs are considered to be working towards quality, while STAR 3 and 4 programs have 
achieved a high-quality designation. 

Refining Keystone STARS 

Keystone STARS rewards participating programs by providing financial and technical supports to improve 
quality. Even with this support, many providers struggle with the costs associated with maintaining quality 
standards. What's more, paperwork burden and challenges in complying with certain STARS standards 
dissuade many providers from participating at all; as of December 2015, less than half [48%) of all open 
certified child care facilities in Pennsylvania had a STAR rating.33 

In light of these challenges, OCDEL announced a "re- visioning" of the STARS system to make it "more 
efficient for providers to participate and more effective at improving child outcomes."34 To this end, OCDEL 
has solicited feedback from providers and the public on the "core principles" to be embedded in the new 
STARS system. These principles, to be released in July 2016, will guide the redevelopment of the QRIS over 
the next year. 


Fiene, R., Greenberg, M., Bergsten, M., Fegley, C., Carl, B., Gibbons, E. (2002). Early Care and Education: They Keystone of Pennsylvania's Future: 
Preparing Our Children for Success. Flarrisburg, PA: The Governor’s Task Force on Early Childhood Care and Education. Retrieved from 
http://prevention.psu.edu/inedia/Drc/files/aualitvfinal.Ddf 

33 Office of Child Development and Early Learning Research. (2015). OCDEL Child Care Providers- December 2015 [Data file]. Retrieved from 
http://www.ocdelresearch.org/Reports/Forms/Allltems.aspx?RootFolder=%2FReports%2FOCDEL%20Public%20Data%20File 

3'* Pennsylvania Early Learning Keys to Quality. (2016). Early Childhood Programs: Re-Visioning Keystone STARS. Retrieved from 
httPs://www.pakevs.org/pages/get.aspx?paee=Refining STARS 


7 


OCDEL's attention to stakeholder engagement and expert feedback from the field is promising as it 
provides community members and childcare providers with the opportunity to inform the system through 
which their programs' quality will be evaluated. It is also essential that revisions to STARS reflect existing 
research on the quality characteristics that matter most for child outcomes. 

Table 1, below, delineates how the current STARS standards correspond with the quality characteristics 
described above. 


Table 1. Keystone STARS Standards & Corresponding Quality Characteristics 


KEYSTONE STARS STANDARDS 

QUALITY CHARACTERISTICS 

Staff Qualifications & Professional Development 

• Staff Credentials & Degrees 

• Professional Development 

Learning Program 

• Adult-Child Interactions 

• Instruction 

• Curriculum 

• Dosage 

• Class Size 

Family & Community Partnerships 

• Family Engagement 

Leadership & Management 

• Director Credentials & Degrees 

• Funding 


In the following section, we highlight practices and challenges under the current requirements of Keystone 
STARS and also identify relevant promising practices from around the country. These opportunities for 
STARS innovation could help refocus support and evaluation on the components that most impact child 
outcomes, such as quality adult-child interactions, professionai deveiopment, and instruction. 

Opportunities for STARS Innovation 

Setting Flexible Standards 

Current practice: 

Like many other QRISs, Keystone STARS relies on a "career lattice" which defines the required levels of 
credential or degree for each staff position in an ECE program. Each STAR level carries different standards 
for each type of staff [e.g. lead teacher, assistant). For example, a STAR 2 center must document that at least 
50% of its lead teachers or group supervisors are at Level V or above on the career lattice [associates 
degree); 50% of their assistant teachers are at Level II or above [3 ECE credits); and 100% of their aides 
are at Level I or above [high school diploma/GED). 

Pennsylvania offers flexibility in how staff can achieve the various levels on the career lattice. Staff can 
achieve a Level VI by earning a Bachelor's degree in early childhood education, by earning a Bachelor's in a 
related field and 30 ECE credits, by earning a Bachelor's in Elementary Education and 18 ECE credits, or by 
earning a Bachelor's in any field and completing the state certification in ECE. 

The chaiienge: 

Keystone STARS offers less flexibility for the actual ECE centers with regard to meeting and sustaining 
staffing requirements at each STAR level. For example, in a small STAR 3 child care center, the loss of one 
credentialed assistant teacher could cause them to drop below the 75% threshold, and drop a full STAR 
level. This is especially problematic given that high levels of staff turnover are common across the field. 
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Promising practice: 

Recognizing that earning a degree is time and cost-intensive, Massachusetts state standards allow center 
directors to either hold the required degree for a given STAR level or demonstrate that a plan is in place to 
meet requirements within five years, This strategy could be adopted and expanded to reward programs 
whose staff are working towards a credential or degree, and to mitigate the challenge of maintaining 
compliance with standards in the midst of staff turnover. Such a strategy could utilize the already existing 
Professional Development Registry to ensure accountability that staff were actively enrolled in courses and 
progressing towards the desired degree. Meanwhile, challenges associated with the cost of course-work for 
staff remain. 

Maximizing investments in Professionai Deveiopment 

Current practice: 

Keystone STARS, like most QRISs, uses credentials and degrees to serve as one proxy for quality of 
instruction and adult-child interactions. Pennsylvania funds a multitude of non-credit-bearing professional 
development opportunities for early childhood providers, which may be recommended or even required. 
This professional development is aligned with STARS standards and designed to increase teacher 
effectiveness, but it does not build towards a credential or degree. 

The challenge: 

Despite research on the importance of professional development, these classes and workshops are not 
accounted for in STARS measurement of staff qualifications. As a result, some veteran early educators and 
program directors are limited from advancing up the career lattice, despite extensive experience, 
participation in PD, and demonstrated impacts in the classroom. The fundamental fact that the quality of a 
teacher's practice is more difficult to measure than the level of formal education has limited many QRIS, 
including STARS, from addressing this challenge. 

Promising practice: 

In acknowledgement of the complexity of measuring staff quality in early childhood education, Delaware 
has created a teacher "portfolio" component which allows educators to translate community-based 
professional development into higher credentials, without taking credit-bearing courses. The portfolio 
gives seasoned teachers an opportunity to demonstrate competencies gained through experience and on- 
going professional development.^^ As Pennsylvania already invests resources in subsidizing community- 
based professional development for STARS providers, the state could experiment with a similar 
competency-based model to leverage existing resources and build out the credentialed work force. 

Supporting Continuing Education 

Current practice: 

Pennsylvania currently provides financial support for staff in centers that participate in Keystone STARS to 
advance their education through the Rising STARS tuition assistance program. The program pays 95% of 
tuition up to $6,000 per year to help off-set the cost of enrolling in credit-bearing college coursework. 


Department of Early Education and Care. (2014). Massachusetts QRIS Standards. Boston, MA: Executive Office of Education. Retrieved from 
http://www.mass.eov/edu/birth-grade-12/earlv-education-and-care/aris/massachusetts-aris-standards.html 
Deiaware Institute for Exceilence in Early Childhood (2016). Early Childhood Credentials. Newark, DE: University of Delaware, College of Education 
& Human Development. Retrieved from https://dieecpd.org/earlv-childhood-credentials 
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The challenge: 

Unfortunately, the program does not cover books, materials, or fees, and is available each year on a first- 
come, first-serve basis. In 2014-15, tuition assistance provided funding to 1,220 early educators, but many 
more would benefit from such support. 

Promising practice: 

Twenty-four states and the District of Columbia currently provide additional scholarships for early 
educators through the Teacher Education and Compensation Program or T.E.A.C.H. program.^^ T.E.A.C.H. is 
a national model that leverages public and private resources to offer financial support for both classes and 
books, and provides ECE staff with raises or bonuses after completion of each successful education year. 
However, states must fully fund it. Pennsylvania once had a T.E.A.C.H. program which demonstrated 
impressive outcomes, but it was eliminated in 2011 due to state budget cuts.^s As Pennsylvania looks to 
expand slots in quality early learning settings, greater investment in tuition assistance and revival of 
T.E.A.C.H. scholarships could help create a more robust pipeline of credentialed staff to support expansion. 

Operationalizing Quality Interactions and Instruction 

Current practice: 

Teacher-student interactions and instruction are arguably both the most important and most difficult to 
measure quality components. Most states' QRIS systems attempt to assess the quality of instruction using 
program assessment tools and environmental rating scales based on classroom observation. Most of these 
tools utilize subscales which measure dimensions of the environment such as the quality of classroom 
interactions, the characteristics of the physical space, and the range of activities offered. 

Keystone STARS uses the ECERS-R environmental rating scale as its primary tool for assessing instruction 
quality and teacher-student interactions. The environmental rating scale score is closely tied to an ECE 
program's STAR level; STAR 3 and 4 programs must pass a specific threshold for each environmental rating 
subscale, and STAR 2 programs must develop an improvement plan if any subscales are below 3.0. 

Including environmental ratings as part of a program's overall QRIS score may be useful. A 2013 meta- 
analysis of four large-scale studies revealed that preschool quality, as measured by the ECERS-R 
assessment, had a moderate but statistically significant association with language and mathematics 
outcomes.39 

The challenge: 

However, ECERS-R is used to operationalize a broad array of environmental components, and a program's 
overall score may be driven as much by their scores on space and furnishings as by interactions and 
instructional quality. Therefore, a number of researchers have called into question the sole use of ECERS-R 
to measure instructional quality. Several studies have also found weak psychometric properties of the 
scales themselves and limited associations with child outcomes.^o 


T.E.A.C.H. Early Childhood National Center (2015). Strategic Opportunities: Exploring Muitivariate Pathways to a Diverse, Weil-Compensated and 
Professional Early Childhood Workforce. Chapel Hill, NC: T.E.A.C.H. Early Childhood National Center. Retrieved from 
http://teachecnationalcenter.org/wp-content/uploads/2 015/ll/T EACH Annual eReport 2015. pdf 
Pennsylvania Partnerships for Children (2011, July 18). T.E.A.C.H. Scholarship Program Funding Ends. Retrieved from 
http://www.papartnerships.org/news/teach-scholarship-program-funding-ends : T.E.A.C.H. Early Childhood Pennsylvania (2010). To Improve His 
Education Invest in Hers. Harrisburg, PA: T.E.A.C.H. Early Childhood Pennsylvania, http:// pacca.org/images/pdf/teach invest 2010.pdf 

Keys, T.D., Farkas, G., Burchinal, M.R., Duncan, G.J., Vandell, D.L., Li, W., Ruzek, E.A. & Howes, C. (2013). Preschool center quality and school 
readiness: Quality effects and variation by demographic and child characteristics. Child Development, 84(4), 1171-1190. 

*0 Gordon, R. A., Fujimoto, K., Kaestner, R., Korenman, S., & Abner, K. (2013). An assessment of the validity of the ECERS-R with implications for 
measures of child care quality and relations to child development. Developmental psychology, 49(1), 146. Retrieved from 
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3681422/ 
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Promising practice: 

Sixteen states use other observational measures in place of or in addition to an ECERS-R. In particular, 
twelve states use the Classroom Assessment Scoring System [CLASS), which exclusively measures teacher- 
child interactions. In Pennsylvania, the CLASS and PAS tools are recommended, but voluntary. While 
adoption of additional measures of instructional quality may boost performance, further research is 
needed. In the meantime, Pennsylvania should be explicit in acknowledging the limitations of current 
measures of instructional quality, and focus on supports that promote research-based practices in the 
classroom. 

Providing Coaching and Technicai Assistance 

Current practice: 

States vary in the type, frequency, and duration of technical support provided to individual programs.^i 
While some provide universal technical assistance as part of participation, others vary the level of supports 
by program need, or only offer it on an as-needed basis. Technical support for participants in Keystone 
STARS is responsive, meaning that providers must request technical assistance for specific quality 
performance standards when they first participate in STARS or to increase their STAR level. Through this 
one-on-one technical assistance, providers develop action plans that are designed to improve programming 
within a particular area. 

The challenge: 

A 2011 analysis of STARS advancement rates demonstrated that programs that received technical 
assistance were twice as likely to move up a STAR level as those who did not receive TA.42 Of course, the 
cost of expanding technical assistance to serve all STAR 1 and 2 centers would be substantial. 

Promising practice: 

Balancing the need for intensive, one-on-one support and its associated cost, Pennsylvania may consider 
leveraging philanthropic dollars or support to individual programs from other organizations. For example. 
United Way's Success By 6 has seen notable success in Philadelphia and similar models could be developed 
for programs interested in increasing program quality. A review of technical assistance in 39 QRISs 
revealed that states utilized a variety of partners for technical assistance including child care resources and 
referral agencies [62%), public agencies [41%), community-based organizations [41%), and university 
faculty or staff [15%).43 

Paying for Quality 

Current practice: 

As of September 2015, 97% of state QRISs provided at least one form of financial incentive,44 including 
tiered subsidy reimbursements; quality-based bonuses, grants, or awards; quality-based tax credits; and 
employee scholarships or school reimbursements. These additional resources can help to boost both QRIS 
participation and program quality. 


Soli, M. (2014). 2014 Fact Sheets: Technical Assistance. Bethesda, MD: Child Trends. Retrieved from 
http://arisnetwork.org/sites/all/files/resources/Debi%20l\/lathias/2015-10-03%2012%3A25/0RIS%20Technical%20Assistance.pdf 

PA Office of Child Development and Early Learning (2013). Research Brief: Volume 2, Issue 2. Retrieved 
from http://arisnetwork.org/sites/all/files/session/resources/OCDEL%20Research%20Brief%20Februarv%202013%20Volume%202%20lssue%202 

.pdf 

*3 Holod, A., Faria, A.M., Weinberg, E. & Howard, E. (2015). Moving Up the Ladder: How Do States Deliver Quality Improvement Supports Within Their 
Quality Rating and Improvement Systems? Washington, DC: American Institutes for Research. Retrieved from 

http://www.air.org/sites/default/files/downloads/report/0ualitv-Rating-and-lmprovement-Svstems-0RIS-Earlv-Childhood-Sept-2015rev.pdf 

Holod, A., Faria, A.M., Weinberg, E. & Howard, E. (2015). Moving Up the Ladder: How Do States Deliver Quality Improvement Supports Within Their 
Quality Rating and Improvement Systems? Washington, DC: American Institutes for Research. Retrieved from 

http://www.air.org/sites/default/files/downloads/report/Oualitv-Rating-and-lmprovement-Svstems-ORIS-Earlv-Childhood-Sept-2015rev.pdf 
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The process for determining the amount of a quality improvement grant varies by state. Pennsylvania takes 
into account program setting, student enrollment, and the level of quality attained by the ECE program. 
Other states base funding on income taxes.^s Keystone STARS also provides centers with quality 
achievement awards that are determined by STAR level, center type [e.g. family home vs. center), and full 
time equivalency.46 For large, STAR 4 centers, the merit award for fiscal year 2015-16 was as high as 
$49,250,47 making it one of the most generous awards of any state QRIS.^^ 

The challenge: 

While merit awards in Pennsylvania are comparatively generous, they are not an entitlement. Providers 
must apply each year, and funding is not adequate to reward all centers who qualify. As a result, providers 
cannot count on merit awards in budgeting for staffing and program costs for the coming year. What's 
more, researchers have found that while program directors see STARS grants and bonuses as essential, 
they are often insufficient for covering extra costs associated with higher quality programming.^^ Early 
education providers face a unique dilemma in that the tuition they charge, whether paid by the family or 
covered by state child care subsidy [for low-income children) does not cover the true cost of care, let alone 
allow for investments in infrastructure or wage increases for highly qualified staff 

For example, a STAR 4 facility in the Philadelphia area receives $8,489 per child in subsidy tuition and 
$1,300 in STARS tiered reimbursement to support quality.so Meanwhile, Philadelphia's Nonprofit Finance 
Fund has estimated the average cost of care per child in a STAR 4 facility at $12,789.51 Providers cannot 
raise prices, as families are already stretched to the limit and many are already priced out altogether. For 
this reason, early education providers are forced to budget very differently than other small businesses, 
often holding limited cash balances and operating month-to-month. It's unsurprising then that directors 
often report using STARS bonuses to help cover basic operating costs, limiting the amount they can devote 
to robust quality improvement. In addition, compliance with STARS standards requires a considerable 
amount of monitoring and paperwork, which itself costs staff time. 

Promising practice: 

Pennsylvania should strive to increase reimbursement rates until they align with the documented true cost 
of care. Given the financial restraints described above, it's essential that the state also work to minimize the 
paperwork burden on STARS-participating providers and maximize the impact of tiered reimbursement 
funds. With research suggesting that children in STAR 3 facilities fare better than children in STAR 2 
facilities on the Work Sampling System assessmenfsz Pennsylvania may consider adopting new systems 
that support moving from a STAR 2 to a STAR 3 level, as well as reimbursing programs after they have 
achieved a STAR 3 rating. For example, in Indiana, providers working to reach the fourth quality level are 
eligible for financial support in advance. Upon reaching the fourth level, providers are also rewarded with a 


Ibid. 

Administration for Chiidren & Families (2014). QRIS Financial Incentives. Washington, DC: Department of Health & Human Services. Retrieved from 
https://arisguide.acf.hhs.gov/files/ORIS Financial Incentives.pdf 
Keystone STARS (2015). Keystone STARS Grant & Award Structure Overview FY 15-16. Retrieved from 
http://www.pakevs.org/uploadedContent/Docs/STARS/STARS Grant Award Structure 15-16.pdf 
Mitchell, A. (2012). Financial Incentives in Quality Rating and Improvement Systems: Approaches and Effects. Retrieved from 
http://www.arisnetwork.org/sites/all/files/resources/gscobb/2012-05- 
24%2015:13/Approaches%20to%20Financial%20lncentives%20in%200RIS.pdf 
Aivarez, K., Epps, A. & Montoya, S. (2015). Overcoming Financial Barriers to Expanding High-Quality Early Care & Education in Southeastern 
Pennsylvania. Phiiadelphia, PA: Nonprofit Finance Fund. Retrieved from http://wiliiampennfoundation.org/sites/defauit/files/reports/Articie.pdf 
so Ibid. 

SI Ibid. 

s 2 Sirinides, P., Fantuzzo, J., LeBoeuf, W., Barghaus, K., & Fink, R. (2015). An inquiry into Pennsylvania's Keystone STARS. Phiiadelphia, PA: 
Consortium for Policy Research in Education. 
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one-time monetary award. ^3 Massachusetts also provides grants that support program improvements to 
increase programs' quality levels; providers may receive funding for either a consistent substitute staff 
member who can oversee programming during staff planning time or may use funds to purchase supplies 
related to a QRIS standard area.34 

Pennsylvania should also consider ways to make financial support for quality improvement more flexible 
for providers. Funding that is currently distributed through grants and awards, which are restricted in their 
use to certain program activities, might be redirected to increase tiered reimbursement. This would not 
only allow programs to use those funds to compensate their staff at a higher rate, but would also tie 
funding to their Child Care Works^s enrollment and thus incentivize more high-quality programs to enroll 
greater numbers of low income children. 

Policymakers should also look to maximize impact by layering funding. For example, policymakers could 
leverage federal dollars for childcare subsidy [which serve low-income children), the state-match [which 
has more flexibility and can serve a broader population), and private dollars, to increase both quality and 
capacity. 


Additional Policy Considerations 

The promising practices outlined above represent opportunities for Pennsylvania to improve and refine 
Keystone STARS as a path towards meaningful quality improvement. To ensure that programs fully benefit 
from an improved QRIS in Pennsylvania, state and local policymakers will need to address a range of policy 
questions in addition to the issues raised above. 

Points of consideration should include: 

• Equity and Inclusivity: Policymakers concerned with equity of access must ensure that resources 
for quality improvement are accessible to ECE providers of all types in urban, suburban, and rural 
communities. In particular, studies have shown that low-income children are more likely than their 
peers to attend group or family child care, as opposed to center or school-based early education 
programs.36 How can policy-makers ensure that QRIS standards are attainable and flexible for family 
and group centers and QRIS supports are responsive to their unique needs? 

• Workforce and Wages: It's well documented that early childhood educators earn significantly less 
than workers of comparable education in other fields,^^ and many continue to rely on public 
benefits themselves.ss A recent federal study found that Pennsylvania preschool teachers, excluding 
those in special education, fell in the lowest bracket for annual median pay, earning $21,930 to 


Administration for Children & Families (2014). QRIS Financial Incentives. Washington, DC: Department of Health & Human Services. Retrieved from 
https://aris guide.acf.hhs.gov/files/ORIS Financial Incentives. pdf 

54 Ibid. 

55 Child Care Works Subsidized Child Care Program helps low-income families pay a portion of their child care fees, and is funded jointly by the state 
and federal government and managed by local Child Care Information Services (CCIS) offices. More information at 
http://www.dhs.pa.gov/citizens/childcareearlvlearning/childcareworkssubsidizedchildcareprogram/ 

55 US Health and Human Services; Administration for Children and Families (2007). National Study of Child Care for Low Income Families Patterns of 
Child Care Use among Low-Income Families. Retrieved from http://www.acf.hhs.gov/sites/default/files/opre/patterns cc execsum.pdf 
5r Whitebook, M. (2014). Building a Skilled Teacher Workforce: Shared and Divergent Challenges in Early Care and Education and in Grades K-12. 
Berkeley, CA: University of California. Retrieved from http://www.irle.berkelev.edu/cscce/wp-content/uploads/2014/09/Building-a-Skilled-Teacher- 
Workforce September-2014 9-25.pdf : Toohey, G. (2016, June 18). Study: Preschool teachers make near-poverty wages in Pa. The Philadelphia 

inquirer. Retrieved from http://www.philly.com/philly/education/20160618_Study Pre-school_teachers_make_near-poverty_wages_in_PA_.html 

55 Whitebook, M., Philips, D., Howes, C. (2014). Worthy Work, STILL Unlivabie Wages; The Early Childhood Workforce 25 Years after the National Child 
Care Staffing Study. Berkeley, CA: University of California. Retrieved from http://www.irle.berkelev.edu/cscce/wp- 
content/uploads/2014/ll/ReportFINAL.pdf 
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$23,890.59 PA child care teachers earned even less; their median pay was only $19,590 annually.^o 
Until this issue is remedied, ECE providers will continue to experience turnover that is destabilizing 
for quality ratings, as well as for the children and families they serve. How can policymakers support 
workforce development and allow providers to increase wages, hire and retain educated staff and 
offer saiaries that are competitive with pubiic schooi districts? 

• Data Quality and Alignment: As previously mentioned, studies on the impact of QRIS are limited 
by challenges that arise in accurately measuring the quality of adult-child interactions and early 
childhood instruction. More research and better instruments are needed to refine the ways in 
which these essential quality components are quantified in QRIS. In addition, while vast amounts of 
data exist on ECE providers, ECE enrollment and outcomes, data are not always accessible to 
researchers or policymakers. These data could help answer lingering questions about the quality 
components that matter most for child outcomes. How can data from various sources be integrated 
to be more informative and actionabie for poiicy-makers? 


Pennsylvania is not alone in facing these tough questions. Across the nation, states who are investing in 
high-quality early education are grappling with similar equity concerns, workforce needs, and data 
limitations. But despite these enduring challenges, Pennsylvania is well-positioned to lead the way in better 
aligning early learning standards and supports for what matters most for child outcomes. 

As the state prepares to revise and strengthen the Keystone STARS Quality Rating and Improvement 
System, policy makers should look to promising practices from other states. The benefits of high quality 
early childhood education have been confirmed by decades of research. Ultimately, wise new investments 
in expanding early education slots can be maximized if they are matched with investments in identifying 
and supporting high quality adult-child interactions and instruction. 
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